R2 artifact registry: host + install native CLI binaries (install.json/install.sh, deps, presign) by Alexgodoroja · Pull Request #32 · pilot-protocol/app-template

Alexgodoroja · 2026-06-22T20:19:55Z

R2 artifact registry — host + install native CLI binaries

Lets the Pilot app store host publisher binaries in Cloudflare R2 and deliver them at pilotctl appstore install. Stacks on #31 (CLI adapter + proc.exec).

What's here

Schema: assets/artifacts (os/arch/url/sha256/unpack/exec_path/deps/order/args).
Build emits install.json + a standalone install.sh (dependency-ordered) into every bundle.
stage.go staging runtime: fetch from R2 → verify sha → stage under $APP (file or tar.gz) → run install args → exec.
Manifest delivery grants (proc.exec, fs.write $APP, net.dial <r2-host>).
publish-server: POST /api/artifact/presign (stdlib SigV4, direct-to-R2 upload) + signing-proxy GET /artifact/....
A/B harness scripts/ab_report.py (used by the CI follow-up).

Validated end-to-end with smol machines (smol-machines/smolvm): real browser upload → submit → build → signed catalogue → pilotctl appstore install → daemon spawn → adapter stages from R2 → pilotctl appstore call boots a real microVM. See docs/R2-PREDEPLOY-REPORT.md.

A/B test results (vanilla CLI vs Pilot app store)

📊 Rendered report: https://raw.githack.com/pilot-protocol/app-template/feat/r2-artifacts-on-cli/docs/samples/ab-report-smolvm.html
(also committed at docs/samples/ab-report-smolvm.html; regenerate with scripts/ab_report.py)

Command	Vanilla (ms)	Pilot (ms)	Δ overhead	Match
Version	60	833	+773	✓
List machines	42	456	+414	✓
Run command in an ephemeral Alpine microVM	5854	5492	−362	✓
Compute in a Python microVM	9374	9501	+127	✓

Adapter overhead is negligible once a VM boot dominates; a fixed ~0.4–0.8s IPC cost on trivial calls. Behavior is identical (exit 0 both ways).

Proof — same Alpine microVM, both paths

# vanilla:  smolvm machine run --net --image alpine -- sh -c "echo …; uname -a; cat /etc/alpine-release"
hello from microVM
Linux container 6.12.87 #1 SMP ... aarch64 Linux
3.24.1

# pilot:    pilotctl appstore call io.pilot.smolvm smolvm.exec '{"args":["machine","run","--net","--image","alpine","--", ...]}'
hello from microVM via pilot
Linux container 6.12.87 #1 SMP ... aarch64 Linux
3.24.1

Adapter-generated smolvm.help (local, no backend)

{
  "app": "io.pilot.smolvm",
  "version": "1.2.0",
  "description": "Run portable, lightweight microVMs (smol machines) from the app store.",
  "duration_classes": {
    "fast": "\u003c~1s — status or cheap call",
    "med": "~1-5s — moderate work",
    "slow": "~5-30s — heavy / multi-step"
  },
    {
      "method": "smolvm.version",
      "kind": "utility",
      "summary": "Print the smolvm version.",
      "duration": "fast"
    },
    {
      "method": "smolvm.exec",
      "kind": "utility",
      "summary": "Run any smolvm subcommand; payload {\"args\":[...]}.",
      "params": {
        "args": "array — verbatim argv forwarded to smolvm"
      },
    {
      "method": "smolvm.help",
      "kind": "meta",
      "summary": "This document — every method with params, kind, and duration class.",
      "duration": "fast",
      "typical_roundtrip": "instant (local, no backend call)"
    }

🤖 Generated with Claude Code

…igned binaries) Adds a Cloudflare R2 artifact registry so the app store can HOST publisher binaries and install them via the generated cli adapter — building on the proc.exec + CLI-adapter work (app-store#24, app-template#31). Supersedes NATIVE-APPS.md's "deliver by reference" model: we now store the bytes in R2. Publisher (publish form Artifacts step) uploads per-OS/arch versioned binaries to R2 and sets install order + optional install args; the Submission carries artifacts[] (url + sha256 + order + args, never bytes). At build, in addition to the signed adapter, the pipeline emits install.json into every bundle tarball and adds the delivery grants (proc.exec, fs.write $APP, net.dial <r2-host>). At install, the adapter's StageAssets() fetches each asset from R2, verifies its sha256, stages it under $APP (single file or tar.gz extracted via host tar with zip-slip defence), runs install args in order, and execs the staged command. - scaffold: Asset schema + validation; install.json generation; stage.go runtime; asset-aware main; manifest delivery grants. - publish: Submission.artifacts[] + validation; install.json shipped in bundle. - e2e: scripts/e2e-smolvm.sh + TestR2AssetDeliveryE2E deliver the real smolvm microVM CLI from R2 and exec it (smolvm --version -> "smolvm 1.2.0"). - docs: R2-ARTIFACT-REGISTRY.md (canonical); NATIVE-APPS.md superseded note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Builds on the R2 artifact delivery with the pieces the full publish→install flow needs, validated end-to-end with smol machines (smolvm) via pilotctl. - deps + ordering: assets gain `name` + `deps`; install order resolved by topological sort (deps override raw `order`), with cycle/unknown-dep validation. - install.sh: build now emits a standalone POSIX install.sh (fetch→sha-verify→ stage→run args, dependency-ordered) alongside install.json, shipped in the bundle. Tested (golden + a live fetch-from-R2 run). - presign upload: publish-server gains POST /api/artifact/presign (stdlib SigV4, no AWS SDK) so the website uploads straight to R2, plus a signing-proxy GET /artifact/... for hosts without a public domain. Live round-trip tested. - docs: R2-DEPLOYMENT.md (plan + PRs), R2-PREDEPLOY-REPORT.md (validation + vanilla-vs-pilot), R2-ARTIFACT-REGISTRY.md updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

scripts/ab_report.py runs equivalent commands two ways — the vanilla CLI and `pilotctl appstore call <id> ...` — capturing each command's output, exit code, and wall-clock time, plus the adapter's generated <ns>.help, and emits a self-contained HTML report. CI runs this per-app on a publish PR (follow-up). docs/samples/ab-report-smolvm.html is a real run for io.pilot.smolvm: identical behavior (microVM boots, exit 0) both ways; adapter overhead is negligible once a VM boot dominates (~400-800ms fixed IPC cost on trivial calls). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…heck ST1013/U1000) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… scrub R2 account id - listing.app_description (long, markdown) now flows to metadata.json description_md (shown by `pilotctl appstore view`); falls back to the one-line description. The short `description` continues to drive the catalogue entry / `appstore list`. - security: remove the hardcoded R2 account id from docs + e2e script (use <ACCOUNT_ID> / require R2_ENDPOINT). No credentials were ever committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- docs/USING-SMOLMACHINES-VIA-PILOT.md — how an agent drives the passthrough, the full command surface (derived from smolvm --help), conventions, and the interactive/serve limitations (the discoverability answer). - docs/PUBLISHING-SMOLMACHINES.md — build-all-artifacts → host → catalogue entry → sign/land runbook; short→list, long→view. - submissions/io.pilot.smolmachines/ — the exact submission (passthrough exec, no enumerated methods) + generated metadata (long description_md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…escription So the store-page Description (long description_md) is the only prose in 'pilotctl appstore view'; the short description stays scoped to the catalogue/list line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… triggers bundle-verify CI) The real submission goes through the publish-server (submit -> build -> approve), not committed to app-template; keep the spec as a sample so the validate workflow (which expects a signed bundle per submissions/<id>/) stays green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Alex Godoroja and others added 3 commits June 22, 2026 11:40

This was referenced Jun 22, 2026

publish: Artifacts step — upload platform binaries to the R2 registry pilot-protocol/website#45

Merged

ci: per-app vanilla-vs-Pilot A/B report on publish PRs #33

Draft

Alex Godoroja and others added 5 commits June 22, 2026 13:37

fix(lint): use http status constants; drop unused keyFromURL (staticc…

3c70c8e

…heck ST1013/U1000) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Alexgodoroja merged commit 5bee7f1 into cli-app-support Jun 22, 2026
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

R2 artifact registry: host + install native CLI binaries (install.json/install.sh, deps, presign)#32

R2 artifact registry: host + install native CLI binaries (install.json/install.sh, deps, presign)#32
Alexgodoroja merged 8 commits into
cli-app-supportfrom
feat/r2-artifacts-on-cli

Alexgodoroja commented Jun 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Alexgodoroja commented Jun 22, 2026